Skip to content

feat: add pyansys-quality-check hook - #486

Draft
Revathyvenugopal162 wants to merge 57 commits into
mainfrom
feat/pyansys-quality-report
Draft

feat: add pyansys-quality-check hook#486
Revathyvenugopal162 wants to merge 57 commits into
mainfrom
feat/pyansys-quality-report

Conversation

@Revathyvenugopal162

@Revathyvenugopal162 Revathyvenugopal162 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

PyAnsys Quality Checks

Project metadata checks

  • PM001: The AUTHORS file exists.
  • PM002: The CHANGELOG.md file exists.
  • PM003: The CODE_OF_CONDUCT.md file exists.
  • PM004: The CONTRIBUTING.md file exists.
  • PM005: The CONTRIBUTORS.md file exists.
  • PM006: The LICENSE file exists.
  • PM007: README exists, with README.rst preferred.
  • PM008: The SECURITY.md file exists.
  • PM009: The .github/CODEOWNERS file exists.
  • PM010: Pyproject.toml references the README file.
  • PM011: Pyproject.toml references the LICENSE file.
  • PM012: Project name follows the ansys-- convention.
  • PM013: Project version follows semantic versioning or accepted dev versions.
  • PM014: Project author and maintainer metadata matches PyAnsys defaults.
  • PM015: The LICENSE file includes recognized project license wording.
  • PM016: The .github/CODEOWNERS file contains at least one valid owner entry.
  • PM017: Project declares supported Python versions with explicit bounds.
  • PM021: The project includes a docs directory.
  • PM022: The project includes a tests directory.
  • PM024: The project supports a task runner such as nox, tox, or pixi.

README checks

  • RM000: README file exists.
  • RM001: README has a PyAnsys badge.
  • RM002: README has a PyPI badge.
  • RM003: README has a Codecov badge.
  • RM004: README has an MIT license badge.
  • RM005: README has a GH-CI badge.
  • RM006: README has an installation section.
  • RM007: README has a documentation section.
  • RM008: README has a license section.

Build system checks

  • BS001: The [build-system] table is declared.
  • BS002: Uses a supported modern build backend.
  • BS003: No legacy setup.py or setup.cfg files are present.
  • BS004: The build backend version is pinned in requires.

CI/CD workflow checks

  • CI001: The ci_cd_main.yml workflow file exists.
  • CI002: The ci_cd_pr.yml workflow file exists.
  • CI003: The ci_cd_release.yml workflow file exists.
  • CI004: Workflows use concurrency blocks.
  • CI005: Workflows set root permissions: {}.
  • CI006: Checkout uses persist-credentials: false.
  • CI007: A labeler job is present across workflows.
  • CI008: The vulnerability check action is used.
  • CI009: The code-style action is used.
  • CI010: The check-pr-title step is present across workflows.
  • CI011: The changelog fragment step is present across workflows.
  • CI012: The doc-style action is used.
  • CI013: The doc-build action is used.
  • CI014: The build-wheelhouse action is used.
  • CI015: The pytest test action is used.
  • CI016: The update-changelog step is present across workflows.

CI/CD file checks

  • CI001: The ci_cd_main.yml workflow file exists.
  • CI002: The ci_cd_pr.yml workflow file exists.
  • CI003: The ci_cd_release.yml workflow file exists.

Dependabot checks

  • DB001: The .github/dependabot.yml file exists.
  • DB002: Dependabot.yml sets version 2.
  • DB003: Pip or uv ecosystem is configured.
  • DB004: The GitHub Actions ecosystem is configured.
  • DB005: A weekly update interval is set.
  • DB006: Cooldown default-days: 7 is configured.
  • DB007: Pip uses the lockfile-only versioning strategy.
  • DB008: Pip groups all dependencies together.

Documentation checks

  • DOC001: The doc/source structure exists.
  • DOC002: The Sphinx config exists.
  • DOC003: The Sphinx config includes numpydoc.
  • DOC004: The Sphinx config includes sphinx_design.
  • DOC005: The Sphinx config includes intersphinx.
  • DOC006: The index page has a getting started section.
  • DOC007: The index page has an API reference section.

Pre-commit checks

  • PC001: The .pre-commit-config.yaml file exists.
  • PC002: Ruff-pre-commit is configured.
  • PC003: Zizmor is configured with the --pedantic flag.
  • PC004: Blacken-docs is configured.
  • PC005: Codespell is configured.
  • PC006: Ansys/pre-commit-hooks is configured.
  • PC007: Google/yamlfmt is configured.
  • PC008: Pyright is configured.
  • PC009: Autofix_prs: true is enabled.
  • PC010: Autoupdate_schedule: weekly is configured.

Security checks

  • SEC001: The .github/zizmor.yml file exists.
  • SEC002: The zizmor config includes the secrets-outside-env rule.
  • SEC003: The gitleaks hook is configured.
  • SEC004: Workflows pin action SHAs.
  • SEC005: SECURITY.md discourages public issue reporting.

Labeler checks

  • LB001: The .github/labeler.yml file exists.
  • LB002: The .github/labels.yml file exists.
  • LB003: Labels.yml has a bug label.
  • LB004: Labels.yml has an enhancement label.
  • LB005: Labels.yml has a documentation label.

Vale checks

  • VL001: The doc/.vale.ini file exists.
  • VL002: Vale uses the Google style package.
  • VL003: Vale uses the ANSYS vocabulary.
  • VL004: The ANSYS accept.txt vocabulary exists.
  • VL005: The ANSYS reject.txt vocabulary exists.

MCP release-readiness checks

  • MCP001: Core governance files are all present.
  • MCP002: All CI/CD workflow files are present.
  • MCP003: The PR workflow wires in a tests job.
  • MCP004: The PR workflow includes a doc-build job.
  • MCP005: README and docs metadata are aligned.
  • MCP006: No TODO or FIXME markers appear in the docs index.
  • MCP007: Security checks are not bypassed.

@github-actions github-actions Bot added the enhancement New features or code improvements label Sep 1, 2026
@RobPasMue

Copy link
Copy Markdown
Member

Just letting you know... I see a lot of overlap with https://github.com/ansys/pre-commit-hooks/blob/main/src/ansys/pre_commit_hooks/tech_review.py

Maybe we can try and merge them into the same hook? Probably bringing in whatever is missing from https://github.com/ansys/pre-commit-hooks/blob/main/src/ansys/pre_commit_hooks/tech_review.py into your hook. Or the other way around

@jorgepiloto jorgepiloto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big addition (new quality-report engine plus 12 rule modules), structure is reasonable, but a few things need fixing before merge.

Request (blocking)

  • .pre-commit-hooks.yaml/setup.py swap out the tech-review hook id and console script instead of adding the new one alongside it. Any config pinned to id: tech-review breaks, and this repo's own .pre-commit-config.yaml:62 still uses it. Keep the old id/script around (deprecated is fine) or call this an intentional breaking change and update the config.

  • project_metadata.py:216, PM010's Poetry-README regex (r"\[^\"']+[\"']") never matches. Tested against readme = "README.rst" under [tool.poetry], it returns None. Every Poetry project fails this check. Reuse the readme in content check from the branch below.

  • project_metadata.py:346-372, PM014's author/maintainer regexes have no closing bound and bleed from authors into maintainers. Tested with a wrong authors name plus a correct maintainers block: both name_ok and email_ok come back True for the wrong author. Bound each regex to its own array, or parse the TOML instead of scanning raw text.

  • pyansys_quality_report.py:552-557, check_dirs_exist gets called twice with identical args in _bootstrap_legacy_files. Copy-paste leftover, remove the second call.

Suggestion

  • pyansys_quality_report.py:774, main() swapped parse_args() for parse_known_args(argv), so a typoed flag now gets dropped silently instead of erroring. Keep it strict unless there's a reason not to.
  • security.py:112-131, SEC004 passes if just one uses: line is SHA-pinned anywhere in the workflow. Compare pinned vs. total count instead.
  • 40+ repeats of "return None if file missing, else file_contains" across dependabot.py, documentation.py, pre_commit.py, labeler.py, vale.py. Worth a checked_contains() helper in common.py.
  • test_pyansys_quality_report.py, three tests os.chdir() into a tmp_path and never chdir back, unlike test_tech_review.py's teardown. Can bite later tests depending on run order.

Comment

  • doc/source/conf.py hides tech_review.py from autoapi rather than removing or deprecating it. Is it staying as dead code on purpose?

Requesting changes: the regex bugs flip pass/fail on real Poetry and author/maintainer configs, and the hook-id removal breaks this repo's own config.

@jorgepiloto jorgepiloto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line-level notes for the earlier review.

Comment thread .pre-commit-hooks.yaml
Comment thread src/ansys/pre_commit_hooks/quality_rules/project_metadata.py
Comment thread src/ansys/pre_commit_hooks/quality_rules/project_metadata.py
Comment thread src/ansys/pre_commit_hooks/pyansys_quality_report.py Outdated
Comment thread src/ansys/pre_commit_hooks/pyansys_quality_report.py
Comment thread src/ansys/pre_commit_hooks/quality_rules/security.py Outdated
Comment thread src/ansys/pre_commit_hooks/quality_rules/dependabot.py Outdated
Comment thread tests/test_pyansys_quality_report.py Outdated
Comment thread doc/source/conf.py Outdated

@jorgepiloto jorgepiloto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have re-reviewed the updates and most prior threads are now addressed.

I am still seeing one blocking backward-compatibility break: .pre-commit-hooks.yaml now exposes only id: pyansys-quality-report and setup.py exposes only the pyansys-quality-report console script, while this repo and existing consumers still use id: tech-review (for example .pre-commit-config.yaml:62). As written, upgrading to this release will break existing pre-commit configs. I suggest keeping tech-review as a compatibility alias (deprecated is fine) until consumers can migrate safely.

@Revathyvenugopal162

Copy link
Copy Markdown
Contributor Author

I have re-reviewed the updates and most prior threads are now addressed.

I am still seeing one blocking backward-compatibility break: .pre-commit-hooks.yaml now exposes only id: pyansys-quality-report and setup.py exposes only the pyansys-quality-report console script, while this repo and existing consumers still use id: tech-review (for example .pre-commit-config.yaml:62). As written, upgrading to this release will break existing pre-commit configs. I suggest keeping tech-review as a compatibility alias (deprecated is fine) until consumers can migrate safely.

reverted in 9c5d29b

@Revathyvenugopal162

Copy link
Copy Markdown
Contributor Author

This is tested in ansys/ansys-sphinx-theme#1123

@jorgepiloto

Copy link
Copy Markdown
Member

Is there any chance we can run a certain set of rules by passing a flag?

I find this mostly interesting in the scenario where we want to perform a technical review using an agent. For example, imagine the following:

prek run tech-review --metadata

Then, the agent reads the output from this tool and applies the required changes.

@Revathyvenugopal162

Copy link
Copy Markdown
Contributor Author

Add the metadata flag, the output will be :
command
pyansys-quality-report --metadata --check PM010,PM014
output

[
  {
    "id": "PM010",
    "family": "project_metadata",
    "name": "Return whether pyproject.toml references the expected README file.",
    "description": "Pyproject.toml references the README file."
  },
  {
    "id": "PM014",
    "family": "project_metadata",
    "name": "Return whether author and maintainer metadata are configured as expected.",
    "description": "Project author and maintainer metadata matches PyAnsys defaults."
  }
]

or for a group/family run:

command
pyansys-quality-report --family documentation --all
output

PyAnsys quality report
========================
Score: 100%
Summary: pass=7 fail=0 warn=0 na=0
- [PASS] DOC001 - The doc/source structure exists.
- [PASS] DOC002 - The Sphinx config exists.
- [PASS] DOC003 - The Sphinx config includes numpydoc.
- [PASS] DOC004 - The Sphinx config includes sphinx_design.
- [PASS] DOC005 - The Sphinx config includes intersphinx.
- [PASS] DOC006 - The index page has a getting started section.
- [PASS] DOC007 - The index page has an API reference section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New features or code improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants